Design Space Exploration of Neural Network Activation Function Circuits

The widespread application of artificial neural networks has prompted researchers to experiment with FPGA and customized ASIC designs to speed up their computation. These implementation efforts have generally focused on weight multiplication and signal summation operations, and less on activation functions used in these applications. Yet, efficient hardware implementations of nonlinear activation functions like Exponential Linear Units (ELU), Scaled Exponential Linear Units (SELU), and Hyperbolic Tangent (tanh), are central to designing effective neural network accelerators, since these functions require lots of resources. In this paper, we explore efficient hardware implementations of activation functions using purely combinational circuits, with a focus on two widely used nonlinear activation functions, i.e., SELU and tanh. Our experiments demonstrate that neural networks are generally insensitive to the precision of the activation function. The results also prove that the proposed combinational circuit-based approach is very efficient in terms of speed and area, with negligible accuracy loss on the MNIST, CIFAR-10 and IMAGENET benchmarks. Synopsys Design Compiler synthesis results show that circuit designs for tanh and SELU can save between 3.13-7.69 and 4.45-8:45 area compared to the LUT/memory-based implementations, and can operate at 5.14GHz and 4.52GHz using the 28nm SVT library, respectively. The implementation is available at: https://github.com/ThomasMrY/ActivationFunctionDemo.Comment: 5 pages, 5 figures, 16 conferenc

AdaBin: Improving Binary Neural Networks with Adaptive Binary Sets

This paper studies the Binary Neural Networks (BNNs) in which weights and activations are both binarized into 1-bit values, thus greatly reducing the memory usage and computational complexity. Since the modern deep neural networks are of sophisticated design with complex architecture for the accuracy reason, the diversity on distributions of weights and activations is very high. Therefore, the conventional sign function cannot be well used for effectively binarizing full-precision values in BNNs. To this end, we present a simple yet effective approach called AdaBin to adaptively obtain the optimal binary sets $\{b_1, b_2\}$ ($b_1, b_2\in \mathbb{R}$) of weights and activations for each layer instead of a fixed set (\textit{i.e.}, $\{-1, +1\}$). In this way, the proposed method can better fit different distributions and increase the representation ability of binarized features. In practice, we use the center position and distance of 1-bit values to define a new binary quantization function. For the weights, we propose an equalization method to align the symmetrical center of binary distribution to real-valued distribution, and minimize the Kullback-Leibler divergence of them. Meanwhile, we introduce a gradient-based optimization method to get these two parameters for activations, which are jointly trained in an end-to-end manner. Experimental results on benchmark models and datasets demonstrate that the proposed AdaBin is able to achieve state-of-the-art performance. For instance, we obtain a 66.4% Top-1 accuracy on the ImageNet using ResNet-18 architecture, and a 69.4 mAP on PASCAL VOC using SSD300. The PyTorch code is available at \url{https://github.com/huawei-noah/Efficient-Computing/tree/master/BinaryNetworks/AdaBin} and the MindSpore code is available at \url{https://gitee.com/mindspore/models/tree/master/research/cv/AdaBin}.Comment: ECCV 202

Data Upcycling Knowledge Distillation for Image Super-Resolution

Knowledge distillation (KD) emerges as a challenging yet promising technique for compressing deep learning models, characterized by the transmission of extensive learning representations from proficient and computationally intensive teacher models to compact student models. However, only a handful of studies have endeavored to compress the models for single image super-resolution (SISR) through KD, with their effects on student model enhancement remaining marginal. In this paper, we put forth an approach from the perspective of efficient data utilization, namely, the Data Upcycling Knowledge Distillation (DUKD) which facilitates the student model by the prior knowledge teacher provided via upcycled in-domain data derived from their inputs. This upcycling process is realized through two efficient image zooming operations and invertible data augmentations which introduce the label consistency regularization to the field of KD for SISR and substantially boosts student model's generalization. The DUKD, due to its versatility, can be applied across a broad spectrum of teacher-student architectures. Comprehensive experiments across diverse benchmarks demonstrate that our proposed DUKD method significantly outperforms previous art, exemplified by an increase of up to 0.5dB in PSNR over baselines methods, and a 67% parameters reduced RCAN model's performance remaining on par with that of the RCAN teacher model

Molecular epidemiology of dengue viruses in southern China from 1978 to 2006

To investigate molecular epidemiology of dengue viruses (DENV) in southern China, a total of 14 dengue isolates were collected in southern China during each epidemic year between 1978 and 2006 and their full-length genome sequences were obtained by using RT-PCR method. The E gene sequences from additional 6 dengue fever patients in Guangzhou in 2006 were also obtained by using RT-PCR method. Combined with DENVs sequences published in GenBank, phylogenetic analysis and recombination analysis were performed. One hundred and twenty-five E gene sequences and 60 complete genome sequences published in the GenBank were also involved. Phylogenetic analysis showed that there was a wide genetic diversity of DENVs isolated in southern China. DENV-1 strains exist in almost all of the clades of genotype I and IV except the Asia 1 clade of genotype I; DENV-2 stains are grouped into four of the five genotypes except American genotype. DENV-4 strains are grouped into 2 genotypes (I and II). Phylogenetic analysis also showed that all DENV-4 isolates and two DENV-2 isolates were closely related to the prior isolates from neighboring Southeast Asia countries. The DENV-1 strain isolated during the 2006 epidemic is highly homologous to the strains isolated during the 2001 epidemic

Microglia exacerbate white matter injury via complement C3/C3aR pathway after hypoperfusion.

Microglial activation participates in white matter injury after cerebral hypoperfusion. However, the underlying mechanism is unclear. Here, we explore whether activated microglia aggravate white matter injury via complement C3-C3aR pathway after chronic cerebral hypoperfusion. Methods: Adult male Sprague-Dawley rats (n = 80) underwent bilateral common carotid artery occlusion for 7, 14, and 28 days. Cerebral vessel density and blood flow were examined by synchrotron radiation angiography and three-dimensional arterial spin labeling. Neurobehavioral assessments, CLARITY imaging, and immunohistochemistry were performed to evaluate activation of microglia and C3-C3aR pathway. Furthermore, C3aR knockout mice were used to establish the causal relationship of C3-C3aR signaling on microglia activation and white matter injury after hypoperfusion. Results: Cerebral vessel density and blood flow were reduced after hypoperfusion (p<0.05). Spatial learning and memory deficits and white matter injury were shown (p<0.05). These impairments were correlated with aberrant microglia activation and an increase in the number of reactive microglia adhering to and phagocytosed myelin in the hypoperfusion group (p<0.05), which were accompanied by the up-regulation of complement C3 and its receptors C3aR (p<0.05). Genetic deletion of C3ar1 significantly inhibited aberrant microglial activation and reversed white matter injury after hypoperfusion (p<0.05). Furthermore, the C3aR antagonist SB290157 decreased the number of microglia adhering to myelin (p<0.05), attenuated white matter injury and cognitive deficits in chronic hypoperfusion rats (p<0.05). Conclusions: Our results demonstrated that aberrant activated microglia aggravate white matter injury via C3-C3aR pathway during chronic hypoperfusion. These findings indicate C3aR plays a critical role in mediating neuroinflammation and white matter injury through aberrant microglia activation, which provides a novel therapeutic target for the small vessel disease and vascular dementia

Health-related quality of life as measured with EQ-5D among populations with and without specific chronic conditions: A population-based survey in Shaanxi province, China

© 2013 Tan et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.Introduction: The aim of this study was to examine health-related quality of life (HRQoL) as measured by EQ-5D and to investigate the influence of chronic conditions and other risk factors on HRQoL based on a distributed sample located in Shaanxi Province, China. Methods: A multi-stage stratified cluster sampling method was performed to select subjects. EQ-5D was employed to measure the HRQoL. The likelihood that individuals with selected chronic diseases would report any problem in the EQ-5D dimensions was calculated and tested relative to that of each of the two reference groups. Multivariable linear regression models were used to investigate factors associated with EQ VAS. Results: The most frequently reported problems involved pain/discomfort (8.8%) and anxiety/depression (7.6%). Nearly half of the respondents who reported problems in any of the five dimensions were chronic patients. Higher EQ VAS scores were associated with the male gender, higher level of education, employment, younger age, an urban area of residence, access to free medical service and higher levels of physical activity. Except for anemia, all the selected chronic diseases were indicative of a negative EQ VAS score. The three leading risk factors were cerebrovascular disease, cancer and mental disease. Increases in age, number of chronic conditions and frequency of physical activity were found to have a gradient effect. Conclusion: The results of the present work add to the volume of knowledge regarding population health status in this area, apart from the known health status using mortality and morbidity data. Medical, policy, social and individual attention should be given to the management of chronic diseases and improvement of HRQoL. Longitudinal studies must be performed to monitor changes in HRQoL and to permit evaluation of the outcomes of chronic disease intervention programs. © 2013 Tan et al.National Nature Science Foundation (No. 8107239

A web-based appointment system to reduce waiting for outpatients: A retrospective study

<p>Abstract</p> <p>Background</p> <p>Long waiting times for registration to see a doctor is problematic in China, especially in tertiary hospitals. To address this issue, a web-based appointment system was developed for the Xijing hospital. The aim of this study was to investigate the efficacy of the web-based appointment system in the registration service for outpatients.</p> <p>Methods</p> <p>Data from the web-based appointment system in Xijing hospital from January to December 2010 were collected using a stratified random sampling method, from which participants were randomly selected for a telephone interview asking for detailed information on using the system. Patients who registered through registration windows were randomly selected as a comparison group, and completed a questionnaire on-site.</p> <p>Results</p> <p>A total of 5641 patients using the online booking service were available for data analysis. Of them, 500 were randomly selected, and 369 (73.8%) completed a telephone interview. Of the 500 patients using the usual queuing method who were randomly selected for inclusion in the study, responses were obtained from 463, a response rate of 92.6%. Between the two registration methods, there were significant differences in age, degree of satisfaction, and total waiting time (<it>P </it>< 0.001). However, gender, urban residence, and valid waiting time showed no significant differences (<it>P </it>> 0.05). Being ignorant of online registration, not trusting the internet, and a lack of ability to use a computer were three main reasons given for not using the web-based appointment system. The overall proportion of non-attendance was 14.4% for those using the web-based appointment system, and the non-attendance rate was significantly different among different hospital departments, day of the week, and time of the day (<it>P </it>< 0.001).</p> <p>Conclusion</p> <p>Compared to the usual queuing method, the web-based appointment system could significantly increase patient's satisfaction with registration and reduce total waiting time effectively. However, further improvements are needed for broad use of the system.</p

Discovering cancer genes by integrating network and functional properties

<p>Abstract</p> <p>Background</p> <p>Identification of novel cancer-causing genes is one of the main goals in cancer research. The rapid accumulation of genome-wide protein-protein interaction (PPI) data in humans has provided a new basis for studying the topological features of cancer genes in cellular networks. It is important to integrate multiple genomic data sources, including PPI networks, protein domains and Gene Ontology (GO) annotations, to facilitate the identification of cancer genes.</p> <p>Methods</p> <p>Topological features of the PPI network, as well as protein domain compositions, enrichment of gene ontology categories, sequence and evolutionary conservation features were extracted and compared between cancer genes and other genes. The predictive power of various classifiers for identification of cancer genes was evaluated by cross validation. Experimental validation of a subset of the prediction results was conducted using siRNA knockdown and viability assays in human colon cancer cell line DLD-1.</p> <p>Results</p> <p>Cross validation demonstrated advantageous performance of classifiers based on support vector machines (SVMs) with the inclusion of the topological features from the PPI network, protein domain compositions and GO annotations. We then applied the trained SVM classifier to human genes to prioritize putative cancer genes. siRNA knock-down of several SVM predicted cancer genes displayed greatly reduced cell viability in human colon cancer cell line DLD-1.</p> <p>Conclusion</p> <p>Topological features of PPI networks, protein domain compositions and GO annotations are good predictors of cancer genes. The SVM classifier integrates multiple features and as such is useful for prioritizing candidate cancer genes for experimental validations.</p